Using a Sequential SOM to Parse Long-term Dependencies
نویسندگان
چکیده
Simple Recurrent Networks (SRNs) have been widely used in natural language processing tasks. However, their ability to handle long-term dependencies between sentence constituents is somewhat limited. NARX networks have recently been shown to outperform SRNs by preserving past information in explicit delays from the network’s prior output. However, it is unclear how the number of delays should be determined. In this study on a shift-reduce parsing task, we demonstrate that comparable performance can be derived more elegantly by using a SARDNET self-organizing map. The resulting architecture can represent arbitrarily long sequences and is cognitively more plausible.
منابع مشابه
Speech Emotion Recognition Using Scalogram Based Deep Structure
Speech Emotion Recognition (SER) is an important part of speech-based Human-Computer Interface (HCI) applications. Previous SER methods rely on the extraction of features and training an appropriate classifier. However, most of those features can be affected by emotionally irrelevant factors such as gender, speaking styles and environment. Here, an SER method has been proposed based on a concat...
متن کاملCompositional Distributional Semantics with Long Short Term Memory
We are proposing an extension of the recursive neural network that makes use of a variant of the long short-term memory architecture. The extension allows information low in parse trees to be stored in a memory register (the ‘memory cell’) and used much later higher up in the parse tree. This provides a solution to the vanishing gradient problem and allows the network to capture long range depe...
متن کاملSelf-organizing maps in sequence processing
Models are abstractions of observed real world phenomena or processes. A good model captures the essential properties of the modeled phenomena. In the statistical learning paradigm the processes that generate observations are assumed unknown and too complex for analytical modeling, thus the models are trained from more general templates with measured observations. A substantial part of the proc...
متن کاملIntegration of remote sensing and meteorological data to predict flooding time using deep learning algorithm
Accurate flood forecasting is a vital need to reduce its risks. Due to the complicated structure of flood and river flow, it is somehow difficult to solve this problem. Artificial neural networks, such as frequent neural networks, offer good performance in time series data. In recent years, the use of Long Short Term Memory networks hase attracted much attention due to the faults of frequent ne...
متن کاملForecasting of rainfall using different input selection methods on climate signals for neural network inputs
Long-term prediction of precipitation in planning and managing water resources, especially in arid and semi-arid countries such as Iran, has a great importance. In this paper, a method for predicting long-term precipitation using weather signals and artificial neural networks is presented. For this purpose, climatic data (large-scale signals) and meteorological data (local precipitation and tem...
متن کامل